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Abstract 

This paper considers a stochastic approximation algo- 
rithm, with decreasing step size and martingale difference 
noise. Under very mild assumptions, we prove the non con- 
vergence of this process toward a certain class of repulsive sets 
for the associated ordinary differential equation (ODE). We 
then use this result to derive the convergence of the process 



when the ODE is cooperative in the sense of fHirsch, 1985 



In particular, this allows us to extend significantly the main 
result of [Hofbauer and Sandholm, 2002| on the convergence 
of stochastic fictitious play in supermodular games. 
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1 Introduction 



Let F : M'' — )■ R'^ be a smooth vector field and {Q, J-", P) a probabil- 
ity space. We consider a M'^-valued discrete time stochastic process 
{xn)n whose general form can be written as the following recursive 
formula: 

Xn+l -Xn = ^^-^ {F{Xn) + Un+l) , (l) 

We assume that {Q, T , P) admits a filtration such that Xq is 

measurable with respect to J-q, and (f/„)„ is a (J-'„)„-adapted se- 
quence of random shocks (or perturbations). Throughout the paper, 
we make the following assumptions: 

Hypothesis 1.1 We assume that: 

(^) {Un)n is a martingale difference: for any n G N*, 

E {Un+l I Tn) = 0. 



(ii) F is Lipschitz continuous, with Lipschitz constant L. 

Such a stochastic approximation process is generally referred to 
as a Robbins-Monro algorithm (see Robbins and Monro, 195T] or 



Kiefer and Wolfowitz, 1952] ). A natural approach to obtain infor- 



mation on the asymptotic behavior of the sample paths {xn{oo))r 
is to compare them to the trajectories of the ordinary differential 
equation 

X = F{x). (2) 

Indeed, one can interpret ([T]) as some kind of Cauchy-Euler approx- 
imation scheme for solving this ODE numerically, with a decreasing 
step size and an added noise. Since we assume that the noise has null 
expectation conditionally to the past, it is natural to expect that, for 
almost every w G fi, the hmit sets of the sample paths (x„(c<;))„ are 
related to the asymptotic behavior of the ODE solution curves. This 
approach was first introduced in [Ljung, 1977| and is usually referred 
to as the ODE method. Thereafter, the method has been studied and 
developed by many authors (including fKushner and Clark, 1978 
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IBenveniste et al., 1990| , |Duflo, 1996| or [Kushner and Yin, 2003| ) 



for very simple dynamics (e.g. linear or gradient-like). 
In a series of papers ( [Benai'm and Hirsch, 1996] and [Benai'm, 1996 



essentially) , Benai'm and Hirsch proved that the asymptotic behavior 
off Xn)n can be described with a great deal of generality through the 
study of the asymptotics of ([2]), regardless of the nature of F. In 
particular, under certain assumptions on the noise, 

(a) the limit sets of {xn)n are almost surely internally chain recur- 
rent in the sense of Bowen and Conley (see [Bowen, 1975 and 



[Conley, 1978| ). This result is detailled in section 2.1. 

(6) the random process (a;„)„ converges with positive probability 
to any given attractor of (|2]). See theorem 7.3 in Benai'm, 1999] 
for a precise statement. 



In addition, it was proved in Pemantle, 1990 that, with probability 



one, {xn)n does not converge to linearly unstable equilibria. Some 
additional references to non convergence results are given in section 
3. 

The motivation of this paper is threefold. First, under some 
additional assumptions on the noise, we prove the non convergence 
of {xn)n toward a certain class of unstable sets (including linearly 
unstable equilibria, periodic orbits and normally hyperbolic sets), 
under less regularity assumptions than the existing results. This is 
detailled in section 3. 

Secondly, in section 4, we use these results, combined with with 
the nature of limit sets (see point (a) above) and the structure of 
chain recurrent sets for cooperative dynamics (see jHirsch, 1999] ) to 
prove convergence of (x„)„ to the set of "stable" equilibria when F 
is cooperative and irreducible. This answers a question raised in 
[Benaim, 2000] . 



Finally, these results are apphed to prove the convergence of stochas- 
tic fictitious play in supermodular games in full generality. This 
proves a conjecture raised in jHofbauer and Sandholm, 2002] . 
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2 Background, Notation and Hypothe- 
ses 



Let F denote a locally Lipschitz vector field on W^. By standard 
results, the Cauchy problem ^ = F{y) with initial condition ?/(0) = 
X admits a unique solution t — )■ defined on an open interval 

Ja; C M containing the origin. For simplicity in the statement of our 
results we furthermore assume that F is globally integrable, meaning 
that Jj; = M for all x G M'^. This holds in particular if F is sublinear; 
that is 

limsup — -j— j- — < oo. 

||a;||— >oo 11-^11 

We let $ = {$t}tgM denote the fiow induced by F. 

A continuous map x '■ ^+ ~^ '^'^ is called an asymptotic pseudo 



trajectory (APT) for $ Benaim and Hirsch, 1996 if, for any T > 0, 



lim dy{t, T) = 0, 

t—^+oo 

where 

d^{t,T)= sup \\x{t + h)-<^Mm\- (3) 

he[0,T] 

In other terms, for any T > 0, the curve joining x{t) to x{t+T) shad- 
ows the trajectory of the semifiow starting from x{t) with arbitrary 
accuracy, provided t is large enough. 

Remark 2.1 Assume that $1 restricted to is uniformly con- 

tinuous. This holds in particular if x or F are bounded maps. Then 

lim dJt, 1) = ^ V T > 0, lim dJt, T) = 0. 

Let {Q, J-", P) be a probability space equipped with some non de- 
creasing sequence of a- algebras {J^t)t>o- Throughout this paper we 
will consider an (J^Jj-adapted continuous time stochastic process 
X = {X{t))t>Q verifying the following condition: 

Hypothesis 2.1 There exists a map co : M.^ — )■ ]R_|_ such that: 

(i) For any 5 > 0, T > 0, 

P (sup dxis, T)>5\Tt] < w(t, 5, T), 
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(ii) limt^oow(t, (5,T) = 0. 

A sufficient condition ensuring hypothesis 12.11 is that 



rt+T 

^{dx{t,T)>6\J^t)< r{s,6,T)ds (4) 



for some r : i— )■ M+ such that 

f oo 



poo 

/ r(s, 5, T)ds < oo. 
Jo 



In this case 

uj{t,5,T)= I r{s,6,T)ds. 



The proof of the following proposition is obvious. 

Proposition 2.2 Under hypothesis \2.1l X is almost surely an asymp- 
totic trajectory for $. 

Example 2.3 (Diffusion processes) 

Let X be solution to the stochastic differential equation 

dX{t) = F{X{t))dt + ^/^dBt, 

where F is a globally Lipschitz vector field, {Bt) a standard Brownian 
motion on Mf^ and 7 : M+ 1— t- a decreasing continuous function. 
Assume that 

^+00 /-c\ 

exp — — dt < +00 
\l{tjj 

for all c > 0. Then (jlj) is satisfied with 

where C and C{T) are positive constants. This is proved in ( [Benai'm, 1999 
Proposition 7.4) 

Example 2.4 (Robbins-Monro algorithms) Let (x„)„ be a stochas- 
tic approximation algorithm governed by the recursive formula 

Xn+l -Xn= 7n+l + f^n+l) , (5) 

where 7„ > 0, X^n^" ~ which satisfies Hypothesis 11.11 As- 

sume furthermore that one of the two following conditions holds: 
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(i) There exists some q >2 such that 

^7^+^/^ < +00 and supE(||t/„r) < +00. 

n 

(ii) (a) The sequence (f/„)„ is suhgaussian (for instance bounded) 

meaning that 

E(exp((0,[/„+i)))|J-„) < exp(rll^ll^) 

for some F > 0; and 
(b) for any c > 0, 



y^exp ( — - I < +00. 



Set r„ := X^ILi 7*- continuous time affine in- 

terpolated process induced by and 7 the piecewise constant 

deterministic process induced by {■yn)n- 

X{Ti + s) := Xi + s^!±i^, for ^ G N, se [0, 7,+i] 

li+i 

and 

7(ri + s) := 7i+i for s G [0,7i+i[. 

Under one of the above condition (i) or (ii), this continuous time 
process is an asymptotic pseudo trajectory of the flow induced by F 
(see Benai'm, 1999] ). Additionally, we have the following result (see 



[Benai'm, 1999 and more specifically [Benai'm, 2000| ): 

Proposition 2.5 Let ko := inf |A; G N | 7a: < -^j- Then, for any 
s > Tkg, condition ^ holds with 

r{s,5,T) = 

in the first case, and 

r{s, 5, T) = 2(iexp ( — 



7(s) 

in the second, where B is some positive constant depending only on 
the noise, the step size and the vector field. 



6 



2.1 The Limit set Theorem 



A set L C M'^ is said to be invariant (respectively positively invari- 
ant) for $ provided C L for all t G M (respectively t G M_|_). 

Let L be an invariant set for We let denote the restriction 
of $ to L. That is, ^t{x) = $t(x) for all x G L and t eR. Note that 
with such a notation $ = ^^"^ . 

An attractor for $^ is a nonempty compact invariant set A G L 
having a neighborhood U in L such that 

lim dist{^f{x),A) = 

t—^oo 

uniformly in x E U. Note that if L is compact, L is always an at- 
tractor for An attractor for distinct from L is called a proper 
attractor. 

The 6asm 0/ attraction of A for is the open set (in L) con- 
sisting of every x G L for which limt_!.oo dist($i(a:), A) = 0. 

A global attractor for $ is an attractor which basin is M.'^. If such 
an attractor exists, $ (respectively F) is called a dissipative flow 
(respectively vector field). 

A compact invariant set L is said to be internally chain-transitive 
or attractor free if $^ has no proper attractor (see e.g. [Conley, 1978| ). 

A fundamental property of asymptotic pseudo trajectories is given 
by the following result due to Benai'm, 1996| for stochastic approxi- 



mation processes and Benai'm and Hirsch, 1996] for APT. We re- 



fer to Benai'm, 1999 for a proof and more details; and also to 



Pemantle, 2007 for a recent overview and some applications. 
Theorem 2.6 Let x be a bounded APT, then its limit set 



is internally chain transitive. 



Corollary 2.7 Under hypothesis \2.1l the limit set of X is almost 
surely internally chain transitive on the event {supj>Q ||X(t)|| < 00}. 
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3 Non convergence toward normally hy- 
perbolic repulsive sets 

From Corollary 12.71 we know that the limit set of X is internally 
chain transitive (ICT). However not every ICT set can be such a 
limit set because the noise may push the process away from cer- 
tain "unstable" sets. For equilibria this question has been tack- 
led by several authors including [Pemantle, 1990| , Tarres, 2001 



Brandiere and Duflo, 1996] and it was proved that, under natural 



conditions, X has zero probability to converge toward a linearly 
unstable equilibrium. This has been extended to linearly unsta- 
ble periodic orbit by [Benai'm and Hirsch, 1995 and to more general 



normally hyperbolic sets by [Benai'm, 1999| ). The proofs of all these 
results rely on the assumption that the unstable manifold of the 
set (to be defined below) is sufficiently smooth (at least C^"^" with 
a > 1/2). While for linearly unstable equilibria or periodic orbit 
such a regularity assumption follows directly from the regularity of 
the vector field, the situation is much trickier for more general sets. 

The purpose of this section is to extend the non convergence 
results mentioned above under less regularity assumptions. This will 
prove to be of fundamental importance in our analysis of cooperative 
dynamics and supermodular games in section |H 

Let 5* be a C^, {d — fc)-dimensional {k G {1, .., d}) submanifold of 
Mf^ and F a compact invariant set contained in S. We assume that 
S is locally invariant meaning that there exists a neighborhood U of 
F in M'^ and a positive time to such that 

^tiuns) c s 

for all \t\ < tQ. We let Q{k,d) denote the Grassman manifold of k 
dimensional planes in M."^. For p & S, the tangent space of S" in p is 
denoted TpS. 

Definition 3.1 F is called a normally hyperbolic repulsive set if 
there exists a continuous map 

peT^ e; egik,d), 

such that 
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(i) for any p G F, 



= TpS © E'j 
(ii) for any t G M and any p G F, 



d<i>,{p)e; = Ei^^^, 

(iii) there exists positive constants A and C such that, for any p G F, 
w G -Ep and t > 0, we have 

\\D<^t{p)w\\ > Ce^*||w||. 

The two basic examples of normally hyperbolic sets are the following. 
For more details, see ( [Benai'm, 1999| , Section 9). 

Example 3.2 (Linearly unstable equilibrium): If F = {p}, 
where p is a linearly unstable equilibrium (not necessarily hyper- 
bolic), then it is a normally hyperbolic repulsive set. 

Example 3.3 (Hyperbolic linearly unstable periodic orbit): 

If F is a periodic orbit, the unity is always a Floquet multiplier. It is 
hyperbolic if the others multipliers all have moduli different from 1 
and it is linearly unstable if at least one has modulus strictly greater 
than one. If both assumptions are checked then F is a normally 
hyperbolic repulsive set. 

For further analysis, it is convenient to extend the map p — )■ E^ to 
a neighborhood of F and to approximate it by a smooth map. More 
precisely it is shown in Benai'm, 1999| , Section 9.1 that there exists 



a neighborhood A/q <Z U of T and a bundle 

= {(p, t;) G 5 n A/'o X M<^ : ve E^} 
where Ep G Q{k,d) such that: 
(i) For aWpeSn Afo, M"' = TpS © E^; 



9 



(ii) the map H : i— )■ M.'^ defined by H{p, v) = p + v induces a 
diffeormorphism from a neigfiborliood of the zero section 
{(p,0) e onto ATq. 

Let now V : Ao ^ M+ be the map defined by V{x) = \\v\\ for 
H~^{x) = {p,v). The form of V implies that there exits L > such 
that 

d{x,S) <V{x) < Ld{x,S) (6) 

for all X G Aq. Then according to Lemma 9.3 in [Benaim, 1999 there 
exist a bounded neighborhood A/i C A/q of F, and numbers T > 0, 
p > 1 such that 

Va; G A/;, V{<^t{x)) > pV{x). (7) 
Given a neighborhood M dU of F we let 

Out, = Out,(A/', S):={xeM\ d{x, 5 n A") > e} . 

and 

In, = s) ■.= ^^\Out,. 



Lemma 3.4 (i) There exists a bounded neighborhood M <Z U of 
F, T > and p > 1 such that for all e > 0, 

$T(Out,(A/', S))nAfc Out^,(A', S). 

In particular, every compact invariant subset contained in M 
lies in S. 

(ii) For all R > there exists a finite set {vi, . . . ,f„} C M'^ and 
a Borel map I : F i— )■ {1, . . . ,n} such that for all p & T and 

V G B{vi(^p), 1), 

p + ev E Outfl,. 

Proof. Choose k E N such that p'^ > L and A/" C Ai be small 
enough so that ^kri-f^) C Afi. Then, using ([6]) and ([7]) for all x E Af, 

d{^kT{x),S) > > ^V{x) > ^d{x,S). 
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Replacing T by kT and p by ^ gives the result. 

We now prove the second assertion. Given i? > 0, let / : F i— )■ 
M*^ be a measurable function such that for all p G T, f{p) G 
and = L{R + 2) where L is the constant appearing in (j6]). 

The bundle E"^ being locally trivial, it is not hard to construct 
such a function. By compactness of /(F), there exists a finite set 
{vi,...,Vn} C /(F) such that /(F) C W^^^B^v,, 1). For j9 G F, set 

I{p) = mm{i = l,...,n : \\f{p) - Vi\\ < 1}. 

Then, for I{p) = i and v G B{vi, 1), 

d{p + efip), S) < d{p + ev, S) + e\\f{p) - v)\\ < d{p + ev, S) + 2e. 

On the other hand, by (E]), 

dip + efip), S) > ^V{p + ef{p)) = '-MM = e(i? + 2). 
Hence 

d{p + ev, S) > Re. 



Corollary 3.5 Let M ,T and p be like in Lemma \3.4l and set 6 



(p — 1) > 0. Let Y be an asymptotic pseudo-trajectory verifying 

(i) x(0) G Out,, 

(ii) for allt> 0,d^{t,T) < Se. 
Then x eventually leaves Af. 

Proof. Suppose that x remains in TV. We claim that xi^T) G Out, 
for all G N. If xikT) G Out, then (^xixikT)) G Outp, by Lemma 
13.41 Hence xi^T + T) G Out, since d^{kT,T) < 6e. This proves the 
claim by induction on k. It follows that the limit set of x meets In, 
but, by the limit set theorem 12.61 and Lemma [3.4[ this limit set has 
to be in S. A contradiction. ■ 
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3.1 Non convergence: sufficient conditions 



Throughout this section we let A/", T and p be hke in Lemma I3.4[ 
and 5 = (p — 1) > 0. We let X be a continuous time (J-t)-adapted 
process verifying hypothesis 12.11 and Et be the event 

Et = {ys>t: X{s) e A/"}. 



Lemma 3.6 On the event {X{t) G Oute}, 

¥{Et\Tt)<io{t,6e,T) 

and 

F{Et\J^t) < I - [I - uj{t + l,6e,T)]F{X{t + 1) e Out,|Ji). 
Proof. The first inequality follows from Corollary 13.51 Now 

¥{Et\Tt)<nEt+i\J^t) 
= ¥{Et+,; X{t + 1) G Out.l j;) + ^Et+i] X{t + 1) G K\J^t) 
= E(P(Et+i|J-t+i)lx(m)eOutJ-^0 +E(P(^t+i|J't+i)lx(m)einJ-^t) 
< uj{t + 1, (5e, T)P(X(t + 1) G Out,| J't) + P(X(t + 1) G K\J^t)- 



Lemma 3.7 Assume that there exists a maps e : ]R_|_ i— t- ]R_|_ with 
limt_5.oo = and constants c > and c' < 1 such that for t large 
enough 

(i) P(X(t + 1) G Out,(t)|J't) > c on the event {X{t) G ln,(t)}. 
(ii) 

oj{t,5e{t),T) < c', 

r/ien 

p(x(t) ^ r) = 0. 
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Proof. One has 



nGN 



and it suffices to prove that P(-En) = for all n G N. 
For all t>n, EnC Et. Thus 

P(^n|JS) < HEtlJ^t) < max(c', 1 - (1 - c')c), 

where the last inequality follows from the assumptions and Lemma 
Now, by a classical Martingale result, 

1 > max(c', 1 - (1 - c')c) > lim F{En\Tt) 1e„ 



almost surely. Hence the result. ■ 

Hypothesis 3.8 Assume that there exists a map 7 
limt^oolit) = and an adapted process (Y(t))t>o such that 

(i) For all e > 0, 

X(t+l)-<l>i(X(t)) 



with 



hm P 



Y{t + r 



0, 



(ii) For all open set O 



liminfP(F(t + 1) G 0\J^t) > 0. 



(iii) There exists a > such that 



lim sup a; (t, a a/7 (t), T) < 1. 

t—^OD 



Theorem 3.9 Let X be a continuous {J^t)-(^do,pted process verifying 
hypotheses \2.1\ and \3.^ Then 



¥{x{t) ^ r) = 0. 
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Proof. We shall prove that the assumptions of Lemma 13.71 are 



fulfilled with e(t) = where a = - and a is given by hypothesis 



iii). Condition (ii) of the lemma is clearly verified. 
To check condition (i) we assume that X(t) e \r\^{t)- Hence (for 
t large enough), $i(X(t)) lies in A/q C and we can write 

$i(X(t))=p(t)+t;(t) 

with {p{t),v{t)) G -Ep^j-) (see the beginning of the section). Then, by 
the triangle inequality, 

d{X{t + l),S) > d{p{t) + ae{t)Y{t + l),S)-\\v{t)\\ 
-ae(t)||F(t + l)-F(t + l)||. 

with 

ae{t) 

Now 

\\v{t)\\ = V(<l>i(X(t))) < Lrf(<l>i(X(t), S)) < Me{t) 

where the first inequality follows from the Lipschitz continuity of the 
map V (see ([6])), and the second from the Lipschitz continuity of $i 
and invariance of S. Thus 

d(X(t + l),S) 



where 



and 



U, 



eit) 

d{p{t) + ae{t)Y{t + l),S) 
e{t) 



Vt = a\\Y{t + l)-Y{t + l)\\. 

Let R = Then by lemma 13.41 (ii) and hypothesis 13.81 (ii) 

there exists c > such that 

> (1 + M)\Tt) = ¥{p{t) + ae{t)Y{t + 1) G OutR^,(t)\J't) > 2c. 

Furthermore, by Hypothesis | 



lim F{Vt > l\Tt) < c 

t—^oo 
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for t large enough. It follows that 

p (^^^^^(^ > 11-^*) > nut -v,>M+ 

> ¥{Ut > 2 + M\J^t) - HVt > l\J^t) > c. 
This proves that condition (i) of the lemma is verified. ■ 

Proposition 3.10 Let X be like in example lKM Setl{t) = \og{'^{t)). 
Assume that 

(i) Function I is sub-additive: l(t + s) < l{t) + l{s). This holds in 

particular if I is concave and l{0) = 

(ii) There exist constants a > b > such that —a <l{t) < —b. 

Then hypothesis \3.8\ holds. In particular, conclusions of Theorems 
IMandlMhold. 

The proof is given in appendix. 

We now apply these results to the specific case of Robbins-Monro 
algorithm. An additional assumption on the noise is needed: 

Hypothesis 3.11 There exists positive real values < < A+ < 
+00 and a continuous map 

Q:M'^^5+ (M'^)n[A-/,,A+/,], 

such that E (^Un+lU^+i \ J^n) = Qi^n)- 

Proposition 3.12 Let {xn)n be a Robbins-Monro algorithm like in 
example 2.4 with 7„ = 1/n and E(||f/„p*' | J^n-i) almost surely 
bounded for some p > 1, which noise also satisfies hypothesis \3.11\ 
Then the associated interpolated process X{t)t>o satisfies Hypothesis 
\3.8\ and therefore, 

¥{x{t) ^ r) = 0. 



Proof. In appendix. 
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4 Application to cooperative dynamics 



Throughout this section we assume that for all x G M'' the Jacobian 
matrix DF{x) = {^{x)) has nonnegative off-diagonal entries and 
is irreducible. Such a vector field F is said to be cooperative and irre- 



ducible Hirsch, 1985 . We refer the reader to Hirsch and Smith, 2006 
for a recent survey on the subject. We furthermore assume that F 
is dissipative, meaning that it admits a global attractor. 

For X, y G W^, X > y means that Xj > i/j for all j. If, additionally, 
X y,we write x > y.lf Xj > yj for all j, it is denoted x ^ y. Given 
two sets A^BdW^ we write A < B provided x < y for aWx E A and 
y E B. Set A is called unordered if for all x,y E A, x < y ^ x = y. 

The vector field F being cooperative and irreducible, its flow has 
positive derivatives Hirsch, 1985 , Hirsch and Smith, 2006] . That 



is D^t{x) ^ for x G M*^ and t > 0. This implies that it is strongly 
monotonic in the sense that (f)t{x) ^ 4't{y) for all x > y and t > 0. 

We let S denote the equilibria set of F. A Point p G is called 
linearly unstable if the Jacobian matrix DF{p) has at least one eigen- 
value with positive real part. We let denote the set of such equi- 
libria and £^ = £ \ £^ . 

An equilibrium point p E £ \s said to be asymptotically stable 
from below if there exists x < p such that — p. The subset of 

equilibria which satisfy this property is denoted £asb- Note that if 
p G £asb, then there exists a non empty open set of initial conditions 
from which the solution trajectories converge to p. In particular £asb 
is countable. Given p G £asb, we introduce the set of points whose 
limit set dominates p: 

V{p) := {x I u{x) > p} 

and we let Sp denotes it boundary: Sp := dV{p). The following 
proposition is basically due to ( ^Hirsch, 1988| , Theorem 2.1) but for 
the regularity proved by [Terescak, 1996| . Our statement follows 



from Proposition 3.2 in [Benai'm, 2000 , where more details can be 
found. 

Proposition 4.1 There exists a unique equilibrium p* G £asb such 
that V{p*) = M'^. For any other p G £asb \ {p*}, Sp is a unordered 
invariant hypersurface diffeomorphic to M*^^^. 
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For p G Sasb \ {p*} we let 7l{^^p) denote the chain recurrent set of 
$ restricted to Sp] or equivalently, the union of all internally chain 
transitive sets contained in Sp. We also set 

7^; = n{<i>^^) \ {s- n Sp}. 



The first part of the next Theorem is proved in Benai'm, 2000 
(see the proof of Theorem 2.1) and the second part restates Theorem 
3.3 in the same paper (relying heavily on [Hirsch, 1999 



Theorem 4.2 For any p e Sasb \ {p*} the set TZ'p is a repulsive 
normally hyperbolic set (in the sense of section l^. Any internally 
chain transitive set is either an ordered arc included in £~ or is 
contained in TZ'^ for some p G £asb \ {p*}- 

Remark 4.3 By a result of [Jiang, 1991)1 , if F is real analytic, it 
cannot have a nondegenerate ordered arc of equilibria 

As a consequence of these results we get the following 

Theorem 4.4 Let X be a continuous (J^t)- (adapted stochastic pro- 
cess verifying hypotheses \2.1\ and \3.^ Then the limit set of X is 
almost surely an ordered arc contained in £~ . In case F is real ana- 
lytic, X{t) converges almost surely to an equilibrium p E . 

Proof. Follows from Theorems 12.61 14.21 and 13.91 ■ 

Corollary 4.5 Let X be the process given in example \2.'J\ with —a < 

m. 

Jit) 



< —b with a> b > 0. Then the conclusions of Theorem \4-.4\ hold 



Corollary 4.6 Let (x„) be the Robbins Monro algorithm given in 
example \2.4\ with '~fn = ji- Assume that hupothesis \3 . 1 1\ holds. Then 



the conclusions of Theorem hold. 



5 Perturbed best response dynamic in 
supermodular games 

5.1 General settings 

Let us consider a N persons game in normal form. Player i's action 
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set is finite and denoted A*, A' is the mixed strategies set: 
A^ := i = {x'{a))^^Ai I x\a) > 0, ^ x'{a) = 1 I 

and : ^4' i-> R his utihty function. The set of action profiles 
(respectively mixed strategy profiles) is denoted A := x^^A* (rcsp. 
A := x^^A*). The utility functions (M*)i=i,..,Ar are defined on A but 
linearly extended to A: 

X = {x\ .., x^) e A ^ u'{x) := J2 u'{a)x\a')...x''{a''). 

a=(ai,..,a^)eA 

We call G{N, A, u) the game induced by these parameters. Through- 
out our study, we assume that agents play repeatedly and indepen- 
dently. By this, we mean that, denoting a„ = (a^, ..,a^) the action 
profile realized at stage n and (J-'„)„ an adapted filtration, we have 



TV 



P (a„+i = {a\ a^) | = J] ^ «+i = I -^n) ■ 



i=l 



For a = (a^, .., a'^), Sa^ denotes the vertex of A* corresponding to the 
pure strategy profile a* and 6a is the extreme point of the polyhedron 
A relative to the pure strategy profile a. At last, Xn is the empirical 
distribution of moves up to time n : 



X. 



n 



1 

m=l 



\ m=l m=l / 



Standing Notation As usual in game theory we let a~* = {a^)j^i, 
x~'^ — {x^)j^i,A^'' — Xj^iA^ etc. We may write (a*,a~') for a — 
{a}, . . . , a^) and so on. 



5.2 Perturbed best response dynamic 

To shorten notation let us take the point of view of player 1. A 
choice function for player 1 is a continuously differentiable map C : 
M^' ^ A^. 
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Let / : M.^ i— t- IR+ be a strictly positive probability density and 
random variable having distribution f{x)dx. We say that 
C is a good stochastic choice function if it is induced by such a 
stochastic perturbation e, in the following sense: for all 11 G 
C (n) is the law of the random variable 

argmax^^^i (n(/3) +£(/?)) . 

A classical example of good stochastic choice function is the Logit 
map: 

Tm\(^\ exp (r/~^n(a)) 

= E,«.exp(,r'n(«)- 

It is induced by a stochastic perturbation with extreme value density 

Fudenberg and Levine, 1998] and [Hofbauer and Sandholm, 2002] ). 



see 



Given a choice function C, the smooth or perturbed best response 
associated to C is the map br^ : A^^ i— )■ defined by 

hr\y) = C{u\- ,y)). 



Definition 5.1 Let br he a perturbed best response for player 1. 
A smooth fictitious play (SFP) strategy induced by br^ is a strategy 
such that, for any other opponent's strategy, 

¥{a\^, = .\Fn) = hv\x-'), (8) 

where is the empirical moves of the opponents up to time n. 

Stochastic fictitious play was originally introduced by Fudenberg 
and Kreps (see [Fudenberg and Kreps, 1993] ) and the concept be- 
hind is that players use fictitious play strategies in a game where 
payoff functions are perturbed by some random variables in the spirit 
of [Harsanyi, 1973| . To be more precise, suppose that at time n + 1, 
the payoff function to player 1 is the map 

M^+i : A h-> M, 

a ^ u^{a) + £:„+i(a^), 
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where e„ G M.^^ is a random vector which conditional law, given 
is f{x)dx. Suppose furthermore that u\_^_^ is known to player 1 as 
well as all the actions oi, . . . , played up to time n. Fictitious play 
assumes that player 1 chooses the best response to x~^. That is 



a. 



argmax^g^iM^+i(/3, 



Hence equation (|8]) holds where br^ is the smooth best response as- 
sociated to the good stochastic choice function induced by £n+iE 
On the subject, see also the papers Fudenberg and Levine, 1995] , 



Fudenberg and Levine, 199 81 or [ Benai'm and Hirsch,T999] . 



Let us get back to the settings described earher with N players. 
We are interested in the asymptotic behavior of the sequence Xn 
when every player adopts a smooth fictitious play strategy. In the 
remaining of the section, an A^-uple of perturbed best response maps 
is given and we let br : A A denote the map defined by 

br(x) := (bri(x-i),..,brN(x-^)) . 

The set of perturbed Nash equilibria, i.e. the set of x G A such that 
br(x) = X (which can be viewed as the Nash equilibria in an auxiliary 
perturbed game) will be refered to as PNE. A simple computation 
gives 

n + 1 

Hence, the expected increments satisfy: 

E {Xn+l - Xn I J^n) = ^ (br(x„) - X„) . 

n + 1 

The recursive formula describing the evolution of the random process 
{xn)n can then be written 

Xn+l =Xn + ^^^Y {F(Xn) + Un+l) , (9) 

where 



^Another approach is to consider that the player chooses to randomize shghtly 
its moves playing a best response relative to a payoff function perturbed by a de- 
terministic map. Hofbauer and Sandholm (see [Hofbauer and Sandholm, 2002| ) 
proved that any admissible stochastic perturbation can be represented in term 
of a deterministic perturbation. 



20 



(i) the vector field F defined by F{x) = br(x) — x is smooth, 

(ii) the noise f/„+i is a bounded martingale difference by construc- 
tion and given by 

Un+i ■■= - br(^„). 

The associated ODE is the perturbed best response dynamic, given 
by 

X = br(x) — X. (10) 

Note that the set of stationary points for this dynamic is exactly 
PNE, the set of perturbed equilibria. Since the vector field F is 
taking values in the tangent space relative to A, TA := xTA* the 
trajectories remain in A. By an obvious abuse of language, we will 
say that a m x m matrix A is positive definite if, for any ( G TA, 
we have 

( ^ ^ C^AC > 0. 

In the following, the set of matrices which are positive definite in 
this sense is denoted 5+(TA). 

Lemma 5.2 Assume that for each i the choice function of player 
i takes values into the interior H o/ A* . Then there exists positive 
values < < < +oo and a continuous function Q : A — ?■ 
5+(TA) n [A-/d,A+/d] such that 

Proof. Let, for x e A and i G {1,...,A^} Q^{x) denote the 
quadratic form on TA' defined by 

Equivalently, Q''{x){C) is the variance of a i— j- {6a, C) under the law 
6r*(x~*). Let Q{x) denote the quadratic form on TA defined by 

N 

1=1 

•^Notice that this property is always satisfied for good stochastic choice func- 
tions 
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Since br^{x~'^)a > and {5a — &r'(x~*) : a G A^} spans TA\ Q^{x) 
is non-degenerate for all i. Hence Q{x) is nondegenerate, and by 
compactness and continuity, there exist > A~ > such that 

A-||Cll'<Q(a:)(C)<A+||C||% vcgta. 

Now 

E ((f/„+if/J,iC,C) I -^n) = Q(x„)(C). 

Hence the result. ■ 

Finally, the discrete stochastic approximation (|9]) is a first case 
Robbins Monro algorithm with g = 2, which satisfies hypothesis 

EH 

5.3 Properties of the best response dynamic for 
super modular games 

We assume here that for each i = 1, . . . ,N the action set A* is 
equipped with a total ordering denoted <; and we focus our at- 
tention on games such that, for a given player, the reward he ob- 
tains by switching to a higher action increases when his opponents 
choose higher strategies. Such games are called supermodular and 
arise in many economic applications : see e.g. [Topkis, 1979] or 
Milgrom and Roberts, 1990| . 

Definition 5.3 We say that the game G{N, A,u) is (strictly) su- 
permodular if, for any pair of distinct players {i,j) and any action 
profiles a = {a^,..,a^) and b = (a^,..,a^) such that a* > 6* and 
a~* = b~^, the quantity {a)— u^{b) is (strictly) increasing in = 6^, 
forj ^ i. 

Remark 5.4 In the particular case where each action set is equal 
to the couple {0, 1}, the state space is the hypercube [0, 1]^ and 
these games have been defined as coordination games in (Benaim 
and Hirch, 1999) 

In the remainder of this section we set = {1, . . . , m*} and we 
assume that < is the natural ordering on integers. For player i, we 
define the invertible linear operator T*: 

^ M™'-\ (x^j,=i,..,„. ^ ((r(xO),),=i,..,m-i 
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with 

k=j+i 

Two mixed strategies can be compared via this operator and T*(x*) < 
T^{y^) if and only if stochastically dominates x^. In the same spirit, 
two strategy profiles can be compared introducing the operator T : 

Naturally, we say that T{x) < T{y) if T^{x') < T\y') for i = l,..,N 
and the order relation relative to T denoted <t in the sequel. The 
following result is proved in [Hofbauer and Sandholm, 2002] . 

Theorem 5.5 (Hofbauer and Sandholm, 2002) Assume that the 
game is strictly supermodular and that every agent plays a smooth 
fictitious play strategy induced by a good stochastic choice function. 
Then 

(i) for i = 1, .., N, 2/"* >T a;"' ^ br*(?/"*) >t br'(a;"*). 

(ii) The conjugate dynami^ is cooperative and irreducible. Hence, 
it is strongly monotone. In particular, z/(x(t))(>o and (y(t))t>o 
solve nOj) with x(0) <t y(0) (and x(0) ^ y(0)j then, for any 

t>o, x(t) <Ty(t), 

(Hi) There exists two perturbed equilibria x <t x such that any 
chain recurrent set relative to the perturbed best response dy- 
namic is included into the interval [x,x], 

Hofbauer and Sandholm then used this theorem combined with re- 
sults from [Benai'm, 2000| to describe the limit set of stochastic fic- 
tions plays for supermodular game. In view of the new results ob- 
tained in this paper and specifically in section H] we are are now 
able to improve notably their results and to prove the convergence 
of stochastic fictions play for supermodular games in full generality. 



■^we refer to the dynamic induced by the conjugation relation T, defined 
on e x,^iR""~i I 1 > i;i > ... > vl^,_^ > Vij and given hy v = 

T{br{T-\v))) -V. 
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Theorem 5.6 Assume that the assumptions of previous theorem are 
satisfied. Then the limit set of {xn)n is almost surely an ordered arc 
of PNE that are not linearly unstable. If we furthermore assume that 
the choice function is real analytic (for instance in the logit case), 
then (xn)n almost surely converges toward a non linearly unstable 
PNE. 

Proof. By Lemma 15.21 and Theorem I5.5[ the conditions to apply 
Corollary 14.61 are met. ■ 

6 Appendix 

6.1 Proof of Proposition 13.101 

The assumptions on 7 easily imply that 

7W >J_>,... 



7(s + t) 7(s) 
Thus 

poo 

u{t, a T)<C exp {-a^e'"C{T)) 

Jo 

and condition (Hi) of hypothesis l3.8h olds. Let 



2^{t + s) 
and let {Y^, s > 0} be solution to 



dY! = AiY! + dB 



t+s 



with initial condition Yq = 0. Condition (i) of Hypothesis [3l8] follows 
from the following lemma. 

Lemma 6.1 

r TII>^ U\^t ^t+S — ^si^t) n ^ IT-N ^ 

lim P( sup II i^. j^=^= — II > e\j^t) = 0. 

t-^oD o<s<i ylit + s) 

In particular, Hupothesis \3.8\ (i) holds with Y(t) = Y^"^ for all t > 1. 
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Proof. Set a{s) = Z\ = - <l>,(Xt) and = a{t + 

s)Z\. Then 

dZl = {F{Xt+s) - F{^s{Xt)))ds + ^/'y{t + s)dBt+, 
= [DFi^,iX,))Zl + oi\\Zl\\)]ds + .f^itT7)dBt+s. 

Hence 

dt' = [DF{<I>,{X,)) + + dB,^., + a{t + .)o(||Z*||), 

a[t + s) 

where o{z) = zrj{z) and hm2^.o'7(^) = ^7(0) = 0. Then 

n*-n*= rAi{Y^-Y:)du+ ra{t + u)o{\\Zi\\)du. 
Jo Jo 

Thus, by Gronwall's inequahty, 

sup iin*-y;*|| <e^i?i 

0<s<l 

with 

Rt = sup a{t + s)o{\\Zl\\) 

0<s<l 



and 



i^' = sup||A*l|<||DF|| + ^. (11) 

s,t ^ 



To conclude the proof it remains to show that 

nRt > sij't) ^ 

as t — )■ oo. 

It follows from the estimate given in example 12.31 that 



P( sup > 5\J^t) < / C7exp( T^)ds < C'exp(- 

o<s<i Jt 7{s) 



lit + l) 



) 



Thus 



n snp a{t + s)\\Zl\\>R\J^,) < P(||^*|| > — ^l-F.)) 
o<s<i a[t+L) 

< Cexp{-R^C{l)). 
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Now, 

P(sup ait + s)\\ZlM\\Zl\\) > 5\:F,) 

0<s<l 

< P( sup a{t + s)\\Zl\\ > R\Ft) + P( sup 77(11^*11) > -^l-^*)- 

0<s<l 0<s<l -K 

< Cexp(-i?2C(l)) + P( sup r/(||Z*||) > ^\7t). 

0<s<l -K 

Since Imiz^o'ni.'^) = 0; 
limsupP( sup a{t + s)\\Zl\\r]{\\Zl\\) > S\Tt) < C exp{-R^C{l)) 

i->oo 0<s<l 

and since R is arbitrary, this proves the result. ■ 

It remains to prove that condition (ii) of hypothesis 13.81 holds. 

Lemma 6.2 Let S be a n x n self-adjoint positive definite matrix 
and 



exp(— i(S ^x,x)) 



v/det(S)(27r)" 

the density of a centered Gaussian vector with covariance S. Let 
< a < (3 respectively denote the smallest and largest eigenvalues 
ofH. Then 

h{x) > i^r^'fM^)- 

Proof. Follows from the estimates det(S) < /3" and (S^^x, x) < 

Mi, m 

a 

Since is a linear function of {Bt+u, < m < s}, it is a Gaussian 
vector under the conditional probability P(-|J-'t). By Ito's formulae, 
its covariance matrix is solution to 

^ + + Id 



ds 



with initial condition Sq = 0; where A** stands for the transpose of 
A*. It is then easy to check that 

S*= [ U\u)U'*{u)du 
Jo 
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where U^{s) is the solution to 

^ = AlU, UiO) = Id. (12) 

Using f|T2|) we see that f/*(s) is invertible and that its inverse {U^{s))~^ 
solves 

^ = -VA"^, V{0) = Id. 
ds 

Using again (IT^ combined with the estimate (ITT]) and Gronwall's 
lemma, we get 

\\U\s)\\ <e^^ 

Similarly 

\\iU\s)y^\\ < e^^ 
It follows that for all vector h, 

e-^'\\h\\ < \\U\s)h\\ < e^'\\h\\. 

Hence 

a\\hf < {Y.\h,h) < h\\hf, 

where a = e~'^^'^du and b = e^^^du. The result then follows 
from Lemma f l6.2p . ■ 

6.2 Proof of Proposition [3321 

Recall that {J^n)n is a given filtration to which the stochastic process 
is adapted. Let := sup{fc G N | r/,. < n} and call {Qn)n the 
sigma algebra (J-m„)n- Let n>l and kn := — m^. We denote 
by the quantity r„„+j- - r„„ (j = 0, .., kn) and t„ := t^^^. Notice 
that \tn-l\ < 7m„- 

For the continuous time interpolated process induced by a discrete 
process hypothesis 13.81 is satisfied if there exists a vanishing 

positive sequence (7(?t,))„ and a ^„-adapted random sequence 
such that 



[i) for any a > 0, 



lim P 

n— >+oo 



n+1 



n] 



> a I a J = 
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(a) for any open set O C M.'^, there exists a positive number S such 
that 

hminf P G O \ Qn) > S almost surely. 

n— >+oo 

(Hi) there exists a > such that 

limsup u{n, oa/ 'j{n),T) < 1. 

n— >+oo 

Let 7(n) := X]fcli7m„+fc- Fii'st, by proposition \2.5\ the map u cor- 
responding to the process (a;„)„ is given by 

B \'^°°^{u)du 
uin,S,T)= ^" ' ■ 

Hence, 

a;(n,av/^,r) < 

Since 

limSUp 2 < +°^' 

the quantity u{n,a\J^~^i{n),T) is smaller than 1, for a large enough. 
The next lemma corresponds to Lemma [6.11 

Lemma 6.3 Point (i) is satisfied for this choice of {'~f{n))n and the 
random sequence {Yn)n given by 



^ -Jl J. I -'IL J. \ 



j=l \k=j+l 



Proof. Set y;+i := "-^+1 ^^^^^ ^ ^ _ 



Then, denoting 



nin+jJ- 
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we have 



-O 



Consequently, 



J — 0, .., kn 1, 



where 



By a recursive argument, 



since = and V^'^"^ Iish+i+i = ^/tM = Cfe-"/^), 

'\/7(™) 

Recall that X]j=i 7mn+i — 1 + 7m„+i and DF is bounded. Conse- 
quently, there exists a real number K such that for n large enough. 



1 



j=l \fc=i+l / 



< e 



sup = e^R 



j=i,..,k 



where Rn ■= ^^Pj=i,..,k„ R^U)- By an application of results 



due to Benaim (see Benai'm, 1999] , proposition 4.1, formula (11) 
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and identity (13) with g = 2), we have 

E ( sup \\Xmr,+j - (pt^iXmnW \ On] < C-f{n). 

\j=0,..,kn-l J 

where C is some positive constant. Additionally, by definition of 

for some function h : M*^_ — >■ R^, strictly increasing and such that 
h{x)/x -^x-^Q+ O'*'. An immediate consequence is that 

< P ( ^ sup h (\\Xm^+j - <j)t^{Xm„)\f \ > a^7H I On J 

< P I sup \\Xm^+j - (ptAXmjf > ("^7H) I Gn J 

Vi=0,..,fc„-1 / 

< : 

- h-^ (a27(n)) ' 

which proves the result. ■ 

To simplify notations, we call E the euclidian space W^. Given n G N, 
the random variable be written .., [/„), where '■ 

{E^, (Be)") {E, Be) is a measurable function. We denote by 
Vu the probability distribution induced by the measurable process 
U = {Un)n ■■ ^) ^ {E^ , {Be)^). We keep the notation Tn for the 
sigma field (Be)'" x E^ when it does not imply any ambiguity. 

Proposition 6.4 There exists a function Pn : (Be)^ x E^ ^ [0, 1] 
called a regular conditional distribution of U given J^n in the sense 
that, for any u G E^ , Pn{-, u) is a probability measure on ((R"')^, {BeY 
and that, for any B G {Be)^), the random variable Pn{B, •) is Tn- 
measurable with 

P„(5, ■) = Fu{B I J^n){-) Pc/ - almost surely. 

For convenience, given u G E^, we denote by PJJ the probability 
measure P„(-,-u) and E"^ the corresponding expectation. Given a 
measurable function y : {E^, (Be)^) — )■ {E,Be), we have 

E-(y) = Euiv I J'n) = Hy{U) I ^„) Fu - a.s.. 
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Lemma 6.5 Letk < i be two natural numbers andy : {E^, (Be)^) — >■ 
{E, Be) be a measurable function. There exists a subset ^o{y) C E^ 
such that Fu{Qo{y)) = 1 and, for any uq G ^o{y), | J^i) and 

^u{y I J^i) CLre Fu-almost surely equal. 

Proof. The random variable z := 'Ku{y \ J-'i) is J-i-measurable. 
Pick a countable 7r-class V such that (j(T>) = J-^. Given A E V, 
there exists a set flo{y,A) such that Fu{flo{y, A)) = 1 and, for any 
Uq G fio{y, A), we have 

(1) {E{lAy I Ti)) = Eu {E{lAy \ J=,) \ J=^) («o), 

(2) E^^{\Ay)^Ev{\Ay\T^){uQ). 

(3) \A^v{y I :Fi) = EuiUy I J'i) Pr-a.s. 

Let us construct QQ{y,A). First, there exist two sets QQ{y,A) 
and Ql{y,A) on which respectively points (1) and (2) arc satisfied 
and such that Fu{Ql{y, Aj) = 1, j = 1,2. Now for the last point, 
one must first consider a set Q^{y,A) such that ¥u{Q,^{y, A)) — 1 
and, for any u e A), 

lA{u)Eu{y I J^i){u) = Eu{lAy I J'i){u) 

Then, by definition of there exists a set f2o(y. A) (which depends 
on fl^{y, A)) such that, Wu{fll{y, A)) = 1 and, for any uq G fll{y, A), 

Fl°in'iy,A)) = Fumy,A) \ Fu){uo) = 1. 

Finally, pick n^iy, A) := A) n Ql{y, A) n Q^Hy, A). 
Now take 

no{y) := nAev^iy,A). 

By countability of V, we have Fu{^o{y)) = 1. There remains to 
prove that, for any uq G fl^oiv), 

I zdFl^^ I ydFl°, for any A G P. 

J A J A 
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= El^' (EuilAV \ J'^)) 

= Eu {EuilAV I J'^) I J'k) (Uo) 

= Eu (JaV I J^k) (mo) 

= KiUy). 

The second equality follows from point (3), the third from point (1) 
and the fifth from point (2). The lemma is proved. ■ 

The following result is due to Hall and Heyde, 1980] (see The- 
orem 3.4 or Theorem 2 page 351 in Chow and Teicher, 1998 for 
a version adapted to our situation). It is a central limit result for 
double arrays. We apply it to prove point {ii). 

Theorem 6.6 (Hall and Heyde) For any n > 1, let kn be a pos- 
itive integer and {Qn, J^"", Pn) o probability space. Consider J-"" C 
J^2 ••• -^k ^ '^'^ increasing family of sigma fields and 
{yj)j=i^,,^kn {J^j')j=i,..,kn- (adapted family of random variables. As- 
sume that 



* for j = l,..,fe 

* we have 



rt; 



En {y^ I -^j^l) = 0, 



^Ej||l^"f I||v.|l>, I J-O 0, 



* there exists a positive, J^i -adapted random sequence {wn)n such 
that 

J]E„ fc%;)^ I J-J^O - 0, 
i=\ 

* there exists a positive random matrix rj, defined on some prob- 
ability space (fi, J-", P), which satisfies 

k 
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Then, denoting yn+i := X]j=i Vj'' sequence {yn)n converges in dis- 
tribution to some random variable y defined on {Q, J-", P) and whose 

characteristic function is given by E ^e~5*^''* j . In particular, 
lim E„ (e'<''y-+^>) = e( e-^'^'") . 

Let us get back to our settings. Let n e N and j G {1, Con- 
sider the measurable functions y^ : {E^, (Be)^) {E, Be), given by 

V7l'"'J \k=j+i J 
where x„ = /i„(mi, Finally, call ?/„ := Yl^iV'j 

Corollary 6.7 Given a nonempty open set O in E, there exist 5 > 
and a set VLq such that P{/(r2o) = 1 and, for any uq eVLq, 

liminf P[/ {y^+i eO\Qn) (mq) > ^- 

n 

Proof. Let VLq be the set 

fl fio [yl lli/;iri||,;ii>.), l|!._,,-*,„(.„j||>.,l,„ 

ngNj=l,..,A:„,rGQ 

By countability, P(f2o) = 1. Pick uq G VLq. We apply Theorem 16.61 to 
(Sin, Tn, Fn) := {E'\ {BeT , P^). = J^m,.+j and the double array 
of random variables {y'j)n,j- 

We now verify that the assumptions required to apply Theorem 
ED hold. First of all 

EZ {y- I J^U) = ^uiy- I J^U) = a.s. 

Secondly, let 

k=j+l 
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A simple computation gives 

g-2pFiu < ||n„,,|| <eii^^ii-. 

Recall that there exists p > 1 such that the sequence of random 
variables (E[/ | J'n-i))n is almost surely bounded. Hence, 
taking q such that 1/p + 1/q — 1 and choosing r e Q, 

-I 2p 

2p 

Consequently, 

A; 

(lb;ll'l||.?ll>^ I -^-i) < C(r) sup 7^;? sup (lkm„+,ir^ I ^-i) 
j=l ^ 

which converges to almost surely. Since uq belongs to the set 
^o(||z/j"||^I||2/«||>r), for any j = 1, ..A;„, 

E^X {\\ymy7\\>r I ^u) = E^^ {\\y7\Wy7\\>r I -^i^i) 

i=i i=i 
and the second point holds. 
From now on, we call 



We have 

1 



7(n) 
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Consequently, 



Let Wn be the i-measurable random variable defined by 



1 



Pick r G Q. By definition of Qq and assumption 12.11 (i), 



sup 

j=l,..k„ 



P ( sup 

Kj=l,..t 

< u(n,r, 1) 



> r 



> r \ Qr 



which implies that 



dist 



n— >+oo 



> 



Since the application Q takes values in [A 1^,1^'^ Id\ and ||n„,j|| 
is bounded above and away from zero, we have 

Wn is a convex combination of such quantities, therefore is bounded. 
Pick some increasing sequence of integers {nk)k- {Wn^)k admits a 
subsequence iWn'^k which converges in distribution to some random 
variable 77"°, defined on the probability space induced by U and 
which takes values in iS"'"(]R'^) fl [ci~/d, a^Id\- 
Now by Theorem 16.61 , 



y< 



with E[/(e*^*'^"''^) = E 2*^''"°*j. In particular, by definition of 

limPc; {vn'^+r G O I GnO) ('«o) = hm P^^? ^ O) = P(y"" G O) >(5, 

where 6 depends on the parameters a and b but not on mq G and 
(n'^)fc. The proof is complete. ■ 
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